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A few calculus rules for chain differentials 


D. E. Clark, J. Houssineau, E. D. Delande 


Abstract 

This paper summarizes the core definitions and results regarding the chain differential for 
functions in locally convex topological vector spaces. In addition, it provides a few elementary 
calculus rules of practical interest, notably for the differentiation of characteristic functionals in 
various domains of physical science and engineering. 


1 Functional differentiation 

In this section we discuss two different forms of differential, the Gateaux differential [5] and the chain 
differential [2], The chain differential, which is similar to the epiderivative [1], is adopted since it is 
possible to determine a chain rule, yet is not as restrictive as the Frechet derivative. 

Results are stated for locally convex topological vector spaces which include Banach spaces such as 
Hilbert and Euclidean spaces, e.g., R n , as well as spaces of test functions for the study of distributions. 
This type of space is therefore sufficiently general for most practical applications. 

1.1 Gateaux differential 

Definition 1 (Gateaux differential). Let X and Y be locally convex topological vector spaces, and let 
Ll be an open subset of X and let f : ff —The Gateaux differential at x G fl in the direction i) G X 
is 

5f(x; 77 ) := lim - (/(x + erf) - f(x)) ( 1 ) 

e->0 e 

when the limit exists. If8f{x\rf) exists for all r/ G X then f is Gateaux differentiable atx. The Gateaux 
differential is homogeneous of degree one in rj, so that for all real numbers a, Sf(x;arj) = a5f(x',rf). 

In Definition 1, the space X might be a function space. In this case, functions on X can be referred 
to as functionals. 

1.2 Chain differential 

Due to the lack of continuity properties of the Gateaux differential, further constraints are required 
in order to derive a chain rule. Bernhard [2] proposed a new form of Gateaux differential defined 
with sequences, which he called the chain differential. It is not as restrictive as the Frechet derivative 
though it is still possible to find a chain rule that maintains the general structure. 

Definition 2 (Chain differential). The function f : X —> Y, where X and Y are locally convex 
topological vector spaces, has a chain differential 8f(x;r/) at point x G X in the direction 77 G X if, for 
any sequence rj m —> 77 G X, and any sequence of real numbers 9 m —> 0, it holds that the following limit 
exists 

8f(x;r]):= lim 2 _ (/( x + 9 m rj m ) - /(x )). ( 2 ) 

m—>oo 

If X = Xi x.. .xX n , where {Xi}" =1 are locally convex topological vector spaces, x := (xi,..., x n ) G X, 
and 7) := (771 ,..., rj n ) G X, the chain differential 5f(x;rj), if it exists, is also called the total chain 
differential of f at point x in the direction r). 
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Definition 3 (n -order chain differential). The n - order chain differential of f at point x in the 
sequence of directions (r?i)™ =1 , is defined recursively with 


6 n f(x-, ( Vi )U) ■■= S(y ^ S n ~ 1 f(y, (%)£?)) (*;»/„). 


(3) 


For the sake of simplicity, when there is no ambiguity on the point at which the chain differential 
is evaluated, the chain differential 5f(x',rj) may also be written as S(f(x);rj). The n th -order chain 
differential (3) then takes the more compact form 

S n f(x; (m)U) ■■= s^-'fix; (Vi)7=i)^n). (4) 

Similarly to the notion of partial derivatives, the notion of chain differential can be defined for 
appropriate multivariate functions. 

Definition 4 (Partial chain differential). Let {Xi }” =1 and Y be locally convex topological vector spaces. 
The function / : x ... x X n —> Y has a partial chain differential 5if{x\ rj) with respect to the i th 

variable, at point x = [x \,..., x n ) £ Xi x ... x X n in the direction rj £ Xi if, for any sequence 
Vm —t V € Xi, and any sequence of real numbers 9 m —> 0 , it holds that the following limit exists 

6if(x;ri):= lim (f(x lt ..., Xi + 0 m r ] m ,..., x n ) - f{x)). (5) 

m —¥oo f7 m 


2 Core calculus rules for the chain differential 

This section summarises the core derivation rules for the chain differential. Note that they all have a 
counterpart defined for the usual derivative. 


Lemma 1 (Chain rule, from [2], Theorem 1). Let X, Y and Z be locally convex topological vector 
spaces, f : Y —x Z , g : X —>• Y and g and f have chain differentials at x in the direction rj and at 
g{x) in the direction 5g(x\if) respectively. Then the composition fog has a chain differential at point 
x in the direction r], given by the chain rule 

$(f°g)(x;v) = Sf(g{x);6g{x;r])). (6) 


Note that, unlike its counterpart for the usual derivative, the chain differential of a composition fog 
does not reduce to the product of a chain differential of / and a chain differential of g. This key dif¬ 
ference has important implications on the structure of the general higher-order chain rule in Theorem 2. 

Similarly to the usual derivatives, the total chain differential of a multivariate function (see Defi¬ 
nition 2 ) can be constructed, in certain conditions defined in the theorem below, as a sum involving 
its partial chain differentials (see Definition 4). 

Theorem 1 (Total chain differential, from [3], Theorem 1). Let {Xj}™ =1 and Y be locally convex 
topological vector spaces, f : X\ x.. ,xX n —> Y, x £ X\ x.. ,xX n and rj := (r/i,..., r] n ) £ X\ x.. .xX n . 
If, for 1 < i < n, it holds that 

1. the partial chain differential 5if exists in a neighbourhood C Xi x ... x X n of x, and, 
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2. the function (x,rj) Sif(x i,... ,Xi-i,x,Xi + i,... ,a : n ;ry) is continuous over fi x JQ, 

then f has a total chain differential at point x in the direction 77 , and it is given by 

n 

Sf{x-ri) = J2 6 if(x\Vi). (7) 

i=l 


The proof is given in appendix in Section A.l. This intermediary result is an important component 
in the construction of the general higher-order chain rule in Theorem 2. 


Theorem 2 (General higher-order chain rule, from [3], Theorem 2). Let X, Y and Z be locally convex 
topological vector spaces, and f :Y —> Z. Assume that g : X —>■ T has higher order chain differentials 
at point x in all the sequences of directions I C {l,...,n}. Assume additionally that there 

exists an open subset Q C Y such that g(x) £ Q and 5^g(x - , ( gf)iei ) £ LI, I C {1,... , ?i}. If, for every 
point y £ 12 and every sequence (£i)i 2 =i £ fl m , 1 < m < n, it holds that 

1. f has a m th -order chain differential at point y in the sequence of directions (£i)i2u? an d> 

2. the conditions of Theorem 1 hold for the function (;</. £1 ..£ m ) 1 —>• S m f(y ; (£i)™ i)> 

3. the functions £j M- 6 m f(y, 1 ), 1 < j < m, are linear and continuous on O, 

then the n th -order chain differential of the composition fog at point x in the sequence of directions 
i s given by 


$ n {f 0 1 ) = ti l7rl f(9{x);(5 H g(x;(ru ) ieu )) w£7r ), ( 8 ) 

7rElI n 

where H n := II({1,..., n}) denotes the set of the partitions of the index set {1,..., n}, and | 7 r| denotes 
the cardinality of the set n. 

The proof is given in appendix in Section A.2. The counterpart for usual derivatives is known as 
Faa di Bruno’s rule [4]. 


Theorem 3 (General higher-order product rule). Let X and Y be locally convex topological vector 
spaces and let g : X —>■ F and z : X —*Y. Assuming that f and g have higher order chain differentials 
at point x in all the sequences of directions ( Vi)iei> I G { 1 ,..., n}, then the product f-g has a n th -order 
chain differential at point x in the sequence of directions (? 7 i)" =1) and it is given by 

5 n if-g){xffg i )U)= E <5 W /(*; (Vi)ie,)S n ~ M g(x-, (^W) ( 9 ) 

7rC{l,...,n} 

where n c := { 1 ,..., n} \ 7 r denotes the complement of n in { 1 ,... ,n}. 

The proof is given in appendix in Section A.3. The counterpart for usual derivatives is known as 
Leibniz’ rule. 


3 Practical derivations for the chain rule 

This section provides specific applications of the chain rule given in Lemma 1 in which the outer function 
of the composition fog assumes a specific form, commonly encountered in practical derivations. 
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Theorem 4 (Practical derivations for the chain rule). Let X, Y and Z be locally convex topological 
vector spaces, f : Y —» Z , g : X —>• Y. Assume additionally that g has a chain differential at some 
point x in some direction 77. Then: 

a) if f is a continuous linear function £, then the composition fog has a chain differential at point 
x in the direction 77 , and it is given by 

6(£og)(x;r]) =t(6g(x;r])). ( 10 ) 


Leibniz’ rule 

b) if f is the k th power function y y k , k > 0 , then the composition fog has a chain differential 
at point x in the direction 77, and it is given by 

S{(y^y k )og)(x;v) = k(g(x)) k 1 6g(x-,rj). ( 11 ) 

c) if f is the exponential function exp ; then the composition fog has a chain differential at point 
x in the direction 77 , and it is given by 

d(exp og)(x; if) =exp {g{x))5g(x\rj). ( 12 ) 

The proof is given in appendix in Section A.4. 


Appendix A Proofs 

A.l Total chain differential (Theorem 1) 

Proof. The result is proved in the case n = 2 from which the general case can be straightforwardly 
deduced. Let us fix a point x = (x,y) € X\ x X 2 and a direction i) = ( 77 , £) e X\ x X 2 , such that / 
has partial chain differentials Sif(x; 77 ) and S 2 f(x; £). Let us then fix arbitrary sequences of directions 
11m — > 77 € Xi, £ m —> £ € X 2 , and an arbitrary sequence of real numbers 0 rn — > 0. For m < 0 we can 
write 


0m 1 [f(x + e m ri) - /(*)] = 6» m 1 [f(x + e m r) m , y + d m f m ) - f(x, y)] 

= ®m [.91 (y + dmf,m) - 9i(y)] + 0~ x [g 2 {x + 9 m i 7m) - g 2 {x)\, 

where we define gi(y) and g 2 (x) as follows: 


(13a) 

(13b) 


9l(y) = f(x + dmVm,y), 

92(x) = f(x, y). 


Given 9. m ^ 0, define h : R —> R as h(t) = gi(y + tf m ). From the mean value theorem for real-valued 
functions, there exists c y € [ 0 , 0 m ] such that 


0)] 


Ah 

^ t=Cy 

Sh(c y ; 1), 


which, when replacing h(t) by gi(y + t£ m ), can be rewritten 

9m [gi(y + 9 m £m) - 9i{y)\ = S(gi{y + Cyfmf 1 ) 

= &9l(y “t" T'yfrn'i f,m)i 


(15a) 

(15b) 

(16a) 

(16b) 
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where Lemma 1 has been used to obtain the last equality. Similarly for g 2 (x), there exists c x € [0, 9 m ] 
such that 

0 ” 1 [g 2 (x + 9 m r] m ) - g 2 {x)\ = Sg 2 (x + c x rj m ; V m ). (17) 

Let us now prove that the limit of the term 


0m 1 [/(* + 9ml 7 ) - /( x)] - S 1 f(x-, rim) - S 2 f(x; £m) 


(18) 


is equal to 0 when r —>• 00 . Substituting (16b) and (17) into (13b), (18) becomes 

\5g 2 (x + c x ri m ;vm) + 5gi{y + CyCmi Cm) ^1 / (*j r]m ) S 2 f[x 5 Cm )|- (19) 

By the triangle inequality, (19) is bounded above by the following summation 

\6g 2 (x + c x Vmi T]m) Vm) | + \&9l ( y + CyCm\ Cm) S 2 f{x;U) |- ( 20 ) 

Substituting gi and g 2 with /, the bound (20) becomes 

|^l /{x “b C x rjmj r]m) (*T> Vm) | “b \^2f {x, y “b Cy£m> f,m) 8 2 f { x : f,m) 1 5 (21) 

which tends to 0 when rn —> oo because of the continuity of the functions (z,u) n- 8if{z,y\v) and 
{z,v) S 2 f(x,z-,u). Thus, it holds that 


lim 

m—> oo 


9m [/(* + 9 m 1 7) - f(x) 


5lf{x\Vm) 


$ 2 /(a:; Cm) 


= 0 , 


( 22 ) 


that is, / has a total chain differential in point x in direction 77 , and it is such that 

8f{x; 77 ) = Sif(x-, 77 ) + S 2 f(x; £), 
which is equivalent to the Proposition 3 in [2], 


(23) 

□ 


A.2 General higher-order chain rule (Theorem 2) 

Proof. The proof is constructed by induction on the number of directions n. Lemma 1 gives the base 
case n = 1. For the induction step, we apply the differential operator to the case n to give the case 
n + 1 and show that it involves a summation over partitions of elements 771 ,, 77 n +i in the following 
way 

8 n+1 (f °9)(x;(Vi)i=i) = S ( u ^ sivl f(9(u);(8 lul g{u;(v j ) jeu )) uew 'C)(x;v n+1 ). (24) 

7ren n ' 2 

The main objective in this proof is to calculate a term of the summation on the right-hand side of 
(24), of the form 

$(u S k f(g(u); (h i (u)) k = 1 )') (x; 1 7 ). (25) 

The additional differentiation with respect to 77 applies to every function on X , i.e. to the function 
g and to the functions hi, 1 < i < k. To highlight the structure of this result, we can define a 
multi-variate function F such that 


F : Y k+1 Z 

(j/o, • • ■, 2 Jk)^ S k f{y 0 ; ( 2 /*)i=i)’ 


(26) 


so that (25) can be rewritten as s(f 0 ^ 771 -^ (g(u),h\{u) 



(a;; 77 ), which is equal to 


SF 



h\{x),.. 


h k (x)-,s(u i-r (g{u),hi{u),. 



(27) 
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using Lemma 1. Let us focus on the direction Syu (g(u), h\{u ),..., h k {u)}j (x\ rj) in (27). Let us 
define a sequence 77 ™ — > 77 £ X and a sequence of real numbers 9 m —> 0. Using the definition of the 
chain differential ( 2 ), we can write 


^ (g(u), hi(u),h k (u ))) (x; 77 ) 

= lim 9m 1 (g{x + 9 m rim),hi(x + 9m'nm),---,hk(x + 9 rn r] m )) - (g(x),hi(x),... ,h k (x)) 

m—> 00 L 

= (Sg(x-, 77), Sfnix; 77 ),, Sh k {x; 77)), 


(28a) 

(28b) 


where the last equality is given by the definition of the chain differential ( 2 ) applied to the functions 
g and hi, 1 < i < k. Substituting (28b) into (27) and applying Theorem 1, (25) becomes 


k 

5 1 F(g{x),hi(x),..., h k {x);5g(x; 77)) + ^ 8i+iF(g(x), hi(x),..., h k (x); Shi(x; 77 )). (29) 

2=1 

• Consider the first term of the summation in (29): 

6 1 F(g(x),h 1 (x),h k (x);Sg(x; 77 )). (30) 

Using the definition of F, it can be written as s(y i-> S k f(y; {hi{x))\ =i)) {g{ x )\ 8g(x i h)) , which 
is equal to 

S k+1 f(g(x); hi(x),..., h k (x),5g(x; 77)) , (31) 

by definition of the (,k + l) t?l -order chain differential. 

• Now consider any other term in (29): 

6 i+ iF(g(x),hi(x),h k (x); Shi(x; 77 )). (32) 

Using the definition of F, it can be written as 

s(y\-^ S k f(g(x);hi(x),. .. ,77, .. .,h k (x))^j (hi(x);8hi{x\ 77 )). (33) 

Let us define a sequence v m —> 5hi(x\ 77) € Y and a sequence of real numbers 9 m —> 0. Using the 
definition of the chain differential (2), (33) becomes 


lim 9 r 

m—t 00 


8 k f(g(x)-,hi(x),..., hi{ x) + 9 m v m ,..., hi(x)) - S k f(g(x);h 1 (x),..., hi(x),..., hi(x)) 


= lim S k f(g(x);h 1 (x),...,iy m ,...,hi(x)) 

m—^oo 

= S k f(g{x); h\(x ),..., 5 hi(x; 77),..., hi(x)), 


(34a) 

(34b) 


while the first equality exploits the linearity of the function 77 S k f (g(x) ; h± (x),..., y,..., h k (x)), 
and the second equality its continuity on ft. 


Substituting (31) and (34b) in (29), (25) becomes 

k 

$ k+ 1 f(g(x); hi(x),..., h k (x),Sg(x-, 77)) + ^ 5 k f(g(x); hi{x), 5hi(x ; 77 ),..., hi 0*0). (35) 

2=1 
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Considering 77 := ? 7 n+i and hi(x) := 5^^g{x-, ( rij)j euJi ) and replacing the result (35) into (24), we 
find 

S n+1 (f o g)(x; 

= E u{{n +1 }}) 

7TGlI n 

+ E Y, 6 M f{s(x);(S^g(x-,(Vj) j eu )) uie ^ }u{ „ u{ „ +1}} ) (36a) 

7TGlI n I'G'TT 

= E <5l7r| /(s , ( a; ); (^ |w| 5(a;;(%)je w )) we7r )- (36b) 

7rElI n +i 

Following a similar argument used for the recursion of Stirling numbers of the second kind and 
their relation to Bell numbers [9, p74], the result above can be viewed as a means of generating all 
partitions of n + 1 elements from all partitions of n elements: The first term in Eq. (36a) corresponds 
to the creation of a new element to the partition 7 r £ H n , containing only n + 1, and each term in the 
second summation appends n + 1 to one of the existing element v of the partition 7r. This argument 
follows similar arguments previously used for ordinary and partial derivatives [ 6 - 8 ]. Hence the result 
is proved by induction. □ 

A.3 General higher-order product rule (Theorem 3) 

Proof. The proof is constructed by induction on the number of directions n. 

a) Case n = 0. 

We can write immediately 


E<5 W /(*; faW)* 0 - 

W g{x; (Vi) ie <D\ tt) = <5 °f(x)6°g(x ) 

(37a) 

7TC 0 

= f{x)g(x) 

(37b) 


= (/ -9){x) 

(37c) 


= S°(f-g)(x). 

(37d) 


b) Case n = 1. 

Let us fix a point x £ X and a direction 771 £ X, such that both / and g have a first-order chain 
differential at point x in direction 771 . Let us then fix a sequence rii,m —>771 € X, and a sequence of 
real numbers 9 m —> 0. Since / has a first-order chain differential at point x, it is continuous in x and 
thus 

lirn f(x + 9 m gi m ) = f(x). (38) 

m—>00 

Since both / and g have a first-order chain differential at point x in direction 771 , we have [2] 


lim 9J- [f(x + 9 m rft m) - f(x)] = 6f{x\ 771 ), (39) 

m—> 00 

lim 0 " 1 [g(x + 6 > m ? 7 i, m ) - g{x)\ = 5g(x ; 771 ). (40) 
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From (38), (39), and (40), it holds that 


lim ®rn [(/ -9)(x + 0mVl,m) ~ (/ ' g)(x )] 

m—>• oo 

= lim 0m[f( x + )g(x + ) /(x + 0 rn T]i jrn )g(^x^ H- f(x-\-0 rn r] hrri )g(x) f{x)g{x)\ 

m—> oo 

(41a) 

= lim f(x + lim O^ 1 [g(x + 0 m 77i, m ) - g(x) 1 

m—>oo m—t oo 

+ lim [/(a: + ^mm.m) -/(a;)] 0 ( 2 :), (41b) 

m—>-oo 

= f(x)5g(x;vi) + $f{x;vi)g(x). (41c) 

That is , f ■ g has a first-order chain differential at point a: in direction rji and it is such that 

&(f ■ g){x; Vi) = f(x)6g(x; in) + Sf(x; rji)g(x). (42) 


c) Case n > 2. 

Let us fix a point x £ X and a sequence of directions (r?i)?= l G X n such that both / and g have higher 
order chain differentials at point x in all sequences of directions ( r]i)iei , I Q {1, ■ • ■ ,n}- We can then 
write 

Y ^'/Oc ( r ?i)ie7i-)^ n_| 7 r| 5( ;c ; (j?»)*e{i,...,n}\ w ) 

7rC{l,...,n} 

= Y S M+1 f(x-, (%)ie7ruw)^" _1_|7r| fi'(a:; (»7i)ie{i,...,n-i}\ w ) 

7rC{l,...,n—1} 

+ <5|7r| /( x ;(%)*e7r)^" _1_|7r|+1 5'(a;;(%)ie{i,...,r l -i}\7ru{n}) (43a) 

7rC{l,...,n—1} 

= 5Z 5(<5 |7r| /(a;;(%)ie7r);%)^" _1_|7r| 5(a:;(r?i)ie{i,...,r l -i}\7r) 

7fC{l,...,n—1} 

+ Y ^ l f(x-,(gi) iev )6(S n ^ 1 ~ M g{x- 1 (n i )ie{i,...,n-i}\ v );Vn) (43b) 

7rC{l,...,n—1} 


= Y 6(6 M f(x;(in)i & ir)6 n 1 M g(x;(in)ie{i,...,n-i}\*);Vn) (43c) 

7 rC{l,...,n—1} 

= <*( Y sM f(x-, (??i)ie 7 r)^" _ 1 _| 7 r| 5 (*; (»?i)ie{i,..,n-i}\ 1 r);»?n) (43d) 

7 rC{l,...,n—1} 

= <5(<5 n_1 (/ • ff)(*; (r/OilTi 1 ); »7n), (43e) 

where the last equality was obtained by exploiting case n — 1. Thus, / • g has a n tft, -order chain 
differential at point x in directions 771 ,..., g n and it is such that 

S n (f-g)(x;(Vi)?= 1 )= 5] S^f(x;( Vi ) ie7r )6 n -^g(x-,(Vi)ie { (44) 

7 rC{l,...,n} 

This ends the proof by induction. □ 


A.4 Practical derivations for the chain rule (Theorem 4) 

Proof. Let us fix a point x & X and a direction i) £ X, such that g has a chain differential at point 
x in direction 77 . Let us then fix a sequence u m —> Sg(x; if) € Y, and a sequence of real numbers 9 m —> 0. 



a) Let us assume that / is a continuous linear function t. For any m > 0 we can write 


Oj [%(a0 + OmVm) - £(g(x))\ = ej [%(x)) + - i(g{x))] (45a) 

= t(y m )- (45b) 

Thus, it holds that 

lim 0^\l(g{x) + O m Vm) ~ t{g{xj)\ = lirn t{v m ) (46a) 

m—¥oo m—¥oo 

= i( lim u m ) (46b) 

= l(bg (x ;??)). (46c) 

Thus, using the definition of the chain differential (2), we have 

Se(g{x);6g(x;ij)) = £(Sg(x;g)). (47) 

Using the chain rule (6) ends the proof. 


b) Let us assume that / is the k th -power function y i-> y k for some k > 0. For any m > 0 we can 
write 


(: g{x ) + 9 m Vm) - ( g(x )) 
fc ' k > 


p—0 


^m 1 (j,) (9( X )) P (°mVm) k P ~ (g(x)Y 


= 6 , 


-1 


k—2 


(g(x)) + k(g(x)) OmVm + ^2 ( ) (g{x)) P (0mVm) k P ~ (#(#)) 


p —0 


= k( g (x)) k + 53 ( k ) (g(x)) p (e m ) k -P- 1 ^ m ) k -P. 
p=o 


Thus, it holds that 


lim 6 

m—>oo 


-1 


(g(x) + 0mVm) k ~ (g{x)Y 


k—2 

= k(g(x)) k 1 lim v m + 'V] ( )(g(x)) P lim (0 m ) fc_p_1 lim ( v m ) k ~ p 

m—t oo z ' ypy m—>00 m—>00 

—Sg(x’,rj) =0 =<5a(au77) fc-p 


=6g(x-,r]) k ~P 


= k(g(x)) k 1 Sg(x;r]). 

Thus, using the definition of the chain differential (2), we have 

Hy ^ y k ){g(x);Sg(x-,r])) = k(g{x)) k ~' L 8g{x\ri). 
Using the chain rule (6) ends the proof. 


(48a) 

(48b) 

(48c) 


(49a) 

(49b) 

(50) 


c) Let us assume that / is the exponential function. For any in > 0 we can write 

^m 1 ex P {g(x) + 0 m u m ) - exp (g(x)) = 9~ x exp (g(x)) [ exp (0 m u m ) - l] 

= d” 1 exp (g(x)) [9 m v m + o(d m v m ) ] ■ 


(51a) 

(51b) 
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Thus, it holds that 


lim 

m—>oo 


exp (g(x) + dmVm) - exp (g(x)) 


exp (g(x)) [ lim v m + lim d m l o(6 m v m ) 


=Sg(x;r)) 


=0 


= exp (g(x))Sg(x; rf). 

Thus, using the definition of the chain differential (2), we have 

5 exp (g(x); 5g{x\ r])) = exp (g(x))6g(x\ rj). 
Using the chain rule (6) ends the proof. 


(52a) 

(52b) 


(53) 

□ 
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